Method and apparatus for reducing image noise

ABSTRACT

An image reduction method and apparatus to digitize and process an image signal. Level values a to h of peripheral pixels of a pattern, the level value of a watched pixel, and the value of a reference level Θ are input. The level value o of the watched pixel is a gain set value α, and added with the level values a to h output from the output ports of a selection circuit and supplied to a divider. The gain set value a is added with a value output from the selection circuit and supplied to the divider. The degree of signal processing is set by a ε-filter and the rate of watched pixels relating to the averaging operation is controlled and thereby, optionally setting the degree of signal processing.

TECHNICAL FIELD

The present invention relates to an image noise reduction method andapparatus for being preferably used to process, for example, an imagesignal by digitizing it. Particularly, the present invention relates toan image noise reduction method and apparatus for solving a trouble whenreducing noise components of an image signal by using the so-calledε-filter.

BACKGROUND ART

For example, to reduce noise components included in an image signal,various methods have been proposed so far. Particularly, one of thesimplest methods having a large noise reduction effect is a method usinga low-pass filter (hereafter referred to as LPF). The LPF is a devicefor transmitting only signals having components lower than a referencefrequency. That is, by inputting a signal whose frequencies change tothe LPF and observing the amplitude of an output signal, acharacteristic is obtained that a component at a higher frequency lowersin level.

However, when viewed from a different point, the LPF uses the averagevalue of a watched pixel and adjacent pixels around the watched pixel asa new value of the watched pixel. That is, in the case of this method,signal levels of watched pixels strongly correlated with peripheralpixels are not greatly changed in their values even if the levels areaveraged. However, random noise components having no correlation areaveraged with noise components included in peripheral pixels andthereby, the value of the component is approached to “0”.

Therefore, when using the above LPF, the noise suppression effectincreases as the search area of peripheral pixels is widened. However,in the case of the averaging operation with peripheral pixels by theLPF, image edge information is reduced similarly to noises andresultantly, the whole image becomes blurry though noises are decreasedand a disadvantage occurs that the image quality is deteriorated.Therefore, an LPF serving as noise reduction means is not generallyused.

To solve the disadvantage of the LPF, the so-called ε-filter isdisclosed (refer to Journal of Institute of Electronics, Information,and Communication Engineers Vol. 77 No. 8, pp. 844–852, April, 1994,Kaoru Arakawa “Nonlinear Digital Filter and Its Application”). That is,in the case of the ε-filter disclosed in this document, when averaging awatched pixel and peripheral pixels, it is first determined whether theperipheral pixels has a correlation with the watched pixel.

Specifically, by setting a certain reference level θ, levels of theperipheral pixels are incorporated into averaging factors when thelevels are included in the range of ±θ of the level of the watched pixelbut they are not incorporated into averaging factors if they are notincluded in the range of ±θ. Thus, whether to incorporate all peripheralfactors into averaging factors is searched and a new value of thewatched pixel is obtained by the averaging operations with the watchedpixel and the peripheral pixels which are regarded as operation objects.

Therefore, even if an image edge enters a search area, when the levelsof pixels constituting the edge exceeds the range of ±θ of the level ofthe watched pixel, the edge is not regarded as an operation object, forexample, it never happens that an image becomes blurry due to pixelsconstituting the edge being included in averaging. That is, with theε-filter, it is possible to suppress only noise components while leavingan image edge as it is by properly selecting the value of he referencelevel θ.

Moreover, an actual circuit configuration of the ε-filter is describedbelow by using FIG. 5. In FIG. 5, the diagram 1 shows a certain onepoint in an image area and imaged states of a watched pixel o and itsperipheral pixels a, b, c, d, e, f, g, and h. Moreover, whensubstituting level values of these pixels with the same notation as a toh and o, the level values a to h of these peripheral pixels are suppliedto a selection circuit 2. Moreover, the value of the above referencelevel θ and the level value o of the watched pixel are input to theselection circuit 2.

In the selection circuit 2, the absolute value (|a−o|) of the differencebetween the level value a of the peripheral pixel a and the level valueo of the watched pixel o is first calculated and the absolute value ofthe difference is compared with the reference level θ. Then, when theabsolute value of the above difference is smaller than the value of thereference level θ, the level value a is output to an output port 3.Moreover, when the absolute value of the difference is larger than thevalue of the reference level θ, the level value a is not output to theoutput port 3 but the value “0” is output. Furthermore, the samecalculations are applied to level values b to h of other peripheralpixels b to h.

Therefore, eight output ports 3 equal to the number of peripheralpixels, for example, are provided for the selection circuit 2, and thelevel values a to h are output to the output ports 3 when the absolutevalue of the above difference is smaller than the value of the referencelevel θ and the value “0” is output to the ports 3 when the absolutevalue of the difference is larger than the value of the reference levelθ. Moreover, an output port 4 is provided for the selection circuit 2and a value obtained by adding “1” to the number of the output ports 3to which the above level values a to h are output is output to theoutput port 4.

That is, level values a to h are output from the output ports 3 of theselection circuit 2 when absolute values of differences between awatched pixel and peripheral pixels are all smaller than the value ofthe reference level θ and the value “9” is output to the output port 4.Moreover, when absolute values of differences between the watched pixeland peripheral pixels are all larger than the value of the referencelevel θ, the value “0” is output from all output ports 3 and the value“1” is output from the output port 4.

Outputs of the output ports 3 of the selection circuit 2 and the levelvalue o of the watched pixel o are supplied to an adder 5 and a valueselected by the output port 6 of the adder 5 is supplied to a divider 7.Moreover, a value outputted from the output port 4 of the selectioncircuit 2 is supplied to the divider 7. Then, in the divider 7, a valueoutputted from the output port 6 of the adder 5 is divided by a valueoutputted from the output port 4 of the selection circuit 2 and thevalue of the above operation result is output by an output port 8.

A certain reference level θ is set, and levels of the peripheral pixelsare incorporated into averaging factors when the levels are included inthe range of ±θ of the level of a watched pixel but the levels are notincorporated into averaging factors when they are not included in therange and then, whether to incorporate all peripheral pixels intoaveraging factors is searched and only peripheral pixels to beincorporated as averaging factors are regarded as operation objects andas a result, a new value of a watched pixel obtained through theaveraging operation with the watched pixel is output to the output port8.

A specific circuit configuration of the selection circuit 2 of the abovedevice is similar to the configuration shown in FIG. 6. That is, in FIG.6, for example, eight comparators 20 equal to the number of the aboveperipheral pixels are obtained. Level values a to h of the aboveperipheral pixels, the level value o of the watched pixel, and the valueof the reference level θ are input to the comparators 20. Then, eachcomparator 20 outputs the value “1” when the absolute value of thedifference between a peripheral pixel and the watched pixel is smallerthan the value of the reference level θ.

Moreover, a signal output from each of the comparators 20 is supplied toan AND gate 21. Furthermore, level values a to h of peripheral pixelsare supplied to the AND gate 21 and corresponding one of the levelvalues a to h of peripheral pixels is output to the output ports 3through the AND gate 21 when a signal output from each of the abovecomparators 20 is equal to “1”. Furthermore, signals output from thecomparators 20 are supplied to an adder 22. Furthermore, an additionoutput of the adder 22 is supplied to an adder 23 and the value “1” isadded and output to the output port 4.

Thereby, in the case of this circuit configuration, level values a to hof peripheral pixels are output through the AND gate 21 when absolutevalues of differences between level values a to h and the level value oof the watched pixel are smaller than the value of the reference levelθ. Moreover, the value “0” is output when absolute values of thedifferences are larger than the value of the reference level θ.Furthermore, a value obtained by adding “1” to the number of levelvalues a to h output to the output ports 3 through the above AND gate 21is output to the output port 4.

Thus, the selection circuit 2 outputs level values a to h when absolutevalues of the above differences are smaller than the value of thereference level θ and a value obtained by adding “1” to the number ofoutput level values a to h. Moreover, the level values a to h and thelevel value o of the watched pixel are added and the addition value isdivided by a value obtained by adding “1” to the number of output levelvalues a to h. Thereby, the averaging operation is applied to onlypixels regarded as averaging factors and a new value of the watchedpixel is derived.

Thus, in the case of the above ε-filter, it is possible to effectivelyreduce noises while preserving an image edges. However, even when usingthe ε-filter, because basic processing is performed by an LPF, imagedetails having a high-frequency component and a small amplitudedisappear. Meanwhile, when an object is flat, noises easily becomeconspicuous and the ε-filter has a large effect. However, when there aremany high-frequency components, the effect of the ε-filter is small andnoises do not easily become conspicuous.

Therefore, when an object has many high-frequency components, theε-filter is turned off. However, it is alternative whether to performsignal processing by the ε-filter or not. In this case, a case occursthat a state in which processing is performed or not performed is notnecessarily the optimum image processing. That is, noises increase whenno operation is performed or image details disappear when an operationis performed depending on the content of an object.

DISCLOSURE OF THE INVENTION

The present invention makes it possible to optionally set the degree ofsignal processing by controlling the rate of watched pixels relating tothe averaging operation by the so-called ε-filter. Therefore, in thecase of the present invention, the averaging operation is performed byweighting level values of watched pixels and optionally controlling therate of the weighting. Image noise reduction method and apparatus of thepresent invention are disclosed correspondingly to the above mentioned.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of an embodiment of aselection circuit used for image noise reduction method and apparatus towhich the present invention is applied.

FIG. 2 is a block diagram showing a configuration of another embodimentof a selection circuit used for image noise reduction method andapparatus to which the present invention is applied.

FIG. 3 is a block diagram for explaining an essential portion of theconfiguration in FIG. 2.

FIG. 4 is illustration for explaining operations of the configuration inFIG. 2.

FIG. 5 is a block diagram for explaining a conventional image noisereduction apparatus.

FIG. 6 is a block diagram showing a configuration of a selection circuitused for conventional image noise reduction method and apparatus.

BEST MODE FOR CARRYING OUT THE INVENTION

The present invention is described below by referring to theaccompanying drawings. FIG. 1 is a block diagram showing a configurationof an image noise reduction apparatus to which image noise reductionmethod and apparatus of the present invention are applied. In FIG. 1, aportion corresponding to that in FIG. 5 is provided with the samesymbol.

In FIG. 1, a pattern 1 shows a certain point in an image area the sameas FIG. 5, which images states of a watched pixel o and its peripheralpixels a, b, c, d, e, f, g, and h. Moreover, when substituting levelvalues of these pixels with the same notations as symbols a to h and o,the level values a to h of these peripheral pixels are supplied to theselection circuit 2. Furthermore, the value of the above reference levelθ and the level value o of the watched pixel are input to the selectioncircuit 2.

In the selection circuit 2, the absolute value (|a−o|) of the differencebetween the level value a of the peripheral pixel a and the level valueo of the watched pixel o is first computed and the absolute value forthe difference is compared with the reference level θ. Then, when theabsolute value of the above difference is smaller than the value of thereference level θ, the level value a is output to the output ports 3.When the absolute value of the difference is larger than the value ofthe reference level θ, the level value a is not output to the outputports 3 but the value “0” is output. The same operation is furtherapplied to level values b to h of other peripheral pixels b to h.

Therefore, for example, eight output ports 3 equal to the number ofperipheral pixels are provided to the selection circuit 2, and the levelvalues a to h are output to these output ports 3 when the absolute valueof the above difference is smaller than the value of the reference levelθ and the value “0” is output to the output ports 3 when the absolutevalue of the difference is larger than the value of the reference levelθ. Moreover, an output port 4 is provided to the selection circuit 2 andthe value of the number of the output ports 3 to which the above levelvalues a to h are output is output to the output port 4. This point isdifferent from the case of FIG. 5.

That is, when absolute values of differences between a watched pixel andperipheral pixels are all smaller than the value of the reference levelθ, the level values a to h are output to the output ports 3 and thevalue “8” is output to the output port 4. Moreover, when absolute valuesof differences between the watched pixel and peripheral pixels are alllarger than the value of the reference level θ, the value “0” is outputfrom all the output ports 3 and the value “0” is output to the outputport 4.

Furthermore, the level value o of the above watched pixel is supplied toa multiplier 9 and an optional gain set value α is supplied to themultiplier 9 and the multiplication of (α×o) is executed. Then, themultiplication value “α×o” derived from the output port 10 of themultiplier 9 is supplied to the adder 5 and added with the level valuesa to h selected by the output ports 3 of the above selection circuit 2.Moreover, an addition value derived from the output port 6 of the adder5 is supplied to a divider 7.

Furthermore, the above gain set value α is supplied to an adder 11 andadded with a value output from the output port 4 of the selectioncircuit 2. Then, the addition value of the gain set value α and thevalue of the number of output ports 3 to which the level values a to hare output, which is obtained by the output port 12 of the adder 11, issupplied to the divider 7. Thus, in the divider 7, a value output fromthe output port 6 of the adder 5 is divided by a value output from theoutput port 12 of the adder 11 and the value of the operation result isderived from the output port 8.

Thereby, when a gain set value α is equal to, for example, 1, theconventional averaging operation is performed and a new value of awatched pixel obtained through the averaging operation is derived fromthe output port 8. However, when assuming a gain set value α as, forexample, 8, the rate of watched pixels relating to the averagingoperation increases, the change of new values of watched pixels derivedfrom the output port 8 is decreased, and a new value close to theoriginal value is derived.

That is, when absolute values of differences between a watched pixel andperipheral pixels are all smaller than the value of the reference levelθ, the value [a+b+c+d+e+f+g+h+α×o] is derived from the output port 6 ofthe adder 5. Moreover, the value [8+α] is derived from the output port12 of the adder 11. Then, in the divider 7, (a+b+c+d+e+f+g+h+α×o)/(8+α)is computed and derived from the output port 8.

Then, for example, when absolute values of differences between a watchedpixel and peripheral pixels are all smaller than the value of thereference level θ and a gain set value α is assumed as 1, a new value of(α+b+c+d+e+f+g+h+o)/(8+1) is derived from the output port 8. In thiscase, the rate of peripheral pixels and a watched pixel in the new valueis 1/9=11.1% of the whole and thereby, image detail components become1/9.

When absolute values of differences between a watched pixel andperipheral pixels are all smaller than the value of the reference levelθ and a gain set value α as 8, a new value of(a+b+c+d+e+f+g+h+8×o)/(8+8) is derived from the output port 8. In thiscase, the rate of peripheral pixels in the new value is 1/16=6.25% whilethe rate of watched pixels is 8/16=50% of the whole.

That is, in this case, by setting a gain set value to 8, the rate of awatched pixel is increased and the noise reduction effect is lowered bya value equivalent to the increased rate but image detail components arepreserved. Thus, by optionally setting a gain set value α in the abovecircuit, it is possible to optionally set the degree of signalprocessing by controlling the rate of a watched pixel relating to theaveraging operation in the so-called ε-filter and thereby, optimum imageprocessing can be executed.

Therefore, for example, a photographer determines the state of an objectand sets a gain set value α to an optional value so that optimum imageprocessing can be performed. Specifically, a gain set value α is set to1 in the case of a flat object such as a beach or sand hill and 8 in thecase of a varied object such as a street corner. When a photographeractually manually sets a gain set value α, it is proper to set it to oneof two stages such as 1 or 8.

However, even when the above setting is performed, the operation of anε-filter is continued. Therefore, even if a gain set value α is set to8, noises are reduced though the effect is deteriorated. This point isgreatly different from the case of the above conventional ε-filter inwhich whether to perform signal processing or not is alternativelyselected. Moreover, when setting includes no signal processing performedby an ε-filter, it is possible to apply a proper signal processing tovarious types of objects.

Therefore, in the case of the above embodiment, because the averagingoperation is performed by weighting level values of watched pixels andoptionally controlling the rate of the weighting level values, it ispossible to optionally set the degree of signal processing bycontrolling the rate of watched pixels relating to the averagingoperation in the so-called ε-filter and thereby, perform optimum imageprocessing.

Consequently, according to the present invention it is possible to solvethe following problem with a conventional apparatus in which whether ornot to perform the signal processing in the so-called ε-filter has beenan alternative, though in that case the state in which signal processingis performed or the state in which signal processing is not performed isnot necessarily said to be an optimum image processing, either.

In the case of the above embodiment, a photographer determines the stateof an object and manually sets a gain set value α. However, it is alsopossible to automatically set a gain set value α by determining theimage of an object through image processing or the like. An embodimentmaking it possible to automatically set a gain set value α is describedbelow.

That is, in this case, the level values a to h of peripheral pixels andthe level value o of a watched pixel are supplied to an α-computingsection 13 as shown in FIG. 2. In the α-computing section 13, thespatial frequency of an image formed by the above watched pixel andperipheral pixels is determined by the α-computing section 13 andmoreover, a gain set value α is computed by determining the distributionof the spatial frequencies. Then, the gain set value α computed by theα-computing section 13 is supplied to the above multiplier 10 and adder11. Other portions are constituted the same as the case of FIG. 1.

Furthermore, a specific configuration of the above α-computing section13 is described below by referring to FIG. 3. However, embodiments ofthe present invention are not limited to this configuration. In FIG. 3,level values of the above peripheral pixels a to h and the watched pixelo are supplied to a spatial high-pass filter (HPF) 100. The high-passfilter 100 detects two-dimensionally how many high-frequency componentsare pre sent in an area when assuming the tap coefficient of the watchedpixel as the value “8” and the tap coefficients of the peripheral pixelsa to h as the value “−1”.

Then, a signal obtained from the output port 101 of the high-pass filter100 is supplied to a conversion-to-absolute-value circuit 102 and asignal obtained from the output port 103 of theconversion-to-absolute-value circuit 102 is supplied to a low-passfilter (LPF) 104. Moreover, a signal obtained from the output port 105of the low-pass filter 104 is supplied to a comparator 106, and iscompared with a reference value optionally set (Reg), and a comparisonoutput is supplied to a control terminal of a selector 107.

Thereby, in the selector 107, when the number of high-frequencycomponents which are spatial frequency components of the peripheralpixels a to h and the watched pixel o increases, the comparison outputof the comparator 106 becomes “H” and a gain set value α1=8 is selected.However, when the number of high-frequency components which are spatialfrequency components of the peripheral pixels a to h and the watchedpixel o decreases, the comparison output of the comparator 106 becomes“L” and a gain set value α2=1 is selected. Then, a selected gain setvalue α is derived from the α-computing section 13.

That is, when, for example, the signal shown in FIG. 4A is input, anoutput of the high-pass filter 100 shows the waveform shown in FIG. 4Band an output of the conversion-to-absolute-value circuit 102 obtainedby converting the signal into an absolute value shows the waveform shownin FIG. 4C. In this case, at the portion of an input signal where thesignal changes are moderate at the left of FIG. 4C, an output of theconversion-to-absolute-value circuit 102 becomes low-level and at theportion on the right side of FIG. 4C where input signal changes areviolent, an output of the conversion-to-absolute value becomes highlevel.

Then, these signals are sent to the low pass filter 104 to derive asignal indicating an envelope curve of the whole level values shown inFIG. 4D, and by comparing the signal by the comparator 106 with thepreset reference value (Reg) it is possible to form a selection signalfor selecting, for example, a gain set value α1 =8 or a gain set valueα2=1 by the selector 107 in accordance with the degree of changes in theinput signal. Thereby, a selected gain set value α is derived from theα-computing section 13.

Therefore, according to this embodiment, a gain set value α isautomatically set and it is possible to eliminate the complexity that aphotographer manually sets the value α. Moreover, according to thisembodiment, it is possible to always change a gain set value α inaccordance with states of the peripheral pixels a to h and the watchedpixel o. For example, it is possible to set an optimum gain set value αfor each portion of an individual object by detecting the portion in onescreen.

Moreover, in the above embodiment, a gain set value α is set to twostages such as the gain set value α1=8 or the gain set value α2=1.However, it is also possible to set a gain set value α in multiplestages by more minutely analyzing the rate of high-frequency componentsin spatial frequency components of the peripheral pixels a to h and thewatched pixel o. Furthermore, though not illustrated, it is possible tocontrol whether or not to perform signal processing by an ε-filter bycontrolling the selection circuit 2 in accordance with a signal outputfrom the α-computing section 13.

Thus, the above image noise reduction method is an image noise reductionmethod of detecting level differences between a watched pixel and itsperipheral pixels, selecting only pixels whose level differences aresmaller than a reference value, and applying the averaging operation tothem, in which the degree of signal processing can be optionally set byweighting the level values of watched pixels and optionally controllingthe rate of weighting level values and performing the averagingoperation and thereby controlling the rate of watched pixels relating tothe averaging operation by the so-called ε-filter and thereby, optimumimage processing can be performed.

Moreover, the above image noise reduction apparatus is an image noisereduction apparatus for reducing noise components, which comprisesdetection means for detecting level differences between a watched pixeland its peripheral pixels, selection means for selecting only pixelswhose level differences are smaller than a reference value, andoperation means for performing the averaging operation by using selectedpixels and which makes it possible to optionally set the degree ofsignal processing by using a means for weighting level values of watchedpixels, controlling the rate of the weighting level values, andperforming the averaging operations by the operation means and therebycontrolling the rate of watched pixels relating the averaging operationsby the so-called ε-filter and thereby, optimum image processing can beperformed.

The present invention is not restricted to the above embodiments but itallows various modifications as long as the modifications are notdeviated from the spirit of the present invention.

That is, according to the present invention, it is possible tooptionally set the degree of signal processing by weighting level valuesof watched pixels, controlling the rate of the weighting level values,performing the averaging operations, and thereby controlling the rate ofwatched pixels relating to the averaging operations by the so-calledε-filter.

Moreover, according to the present invention, it is possible to performoptimum image processing in accordance with the image of an object bycontrolling weighting in accordance with the image of the object.

Furthermore, according to the present invention, it is possible toeliminate the complexity that a photographer manually sets a gainbecause the gain is automatically set by determining the spatialfrequency of an image formed by a watched pixel and its peripheralpixels and controlling weighting in accordance with the abovedetermination result.

Furthermore, according to the present invention, it is possible toperform very preferable processing by digitizing and processing eachpixel level.

Furthermore, according to the present invention, it is possible tooptionally set the degree of signal processing by weighting level valuesof watched pixels, optionally controlling the rate of the weightinglevel values, performing the averaging operations, thereby controllingthe rate of watched pixels relating to the averaging operations by theso-called ε-filter and thereby, optimum image processing can beperformed.

Furthermore, according to the present invention, it is possible toperform optimum image processing in accordance with the image of anobject by using a means for controlling weighting in accordance with theimage of the object.

Furthermore, according to the present invention, it is possible toeliminate the complexity that a photographer manually sets a gainbecause the gain is automatically set by using means for determining thespatial frequency of an image formed by a watched pixel and itsperipheral pixels and control means for controlling weighting inaccordance with the above determination result.

Furthermore, according to the present invention, it is possible toperform very preferable processing by digitizing and processing eachpixel level.

Thereby, a conventional apparatus alternatively selects whether or notto perform signal processing by the so-called ε-filter. In this case, acase occurs in which a state performing or not performing processing isnot necessarily the optimum image processing. However, the presentinvention can preferably solve the problems.

1. An image noise reduction method for reducing a noise component,comprising: detecting level differences between a watched pixel andperipheral pixels; selecting pixels having said level differencessmaller than a reference value to perform averaging operations;weighting the level values of said watched pixel; controlling the rateof said weighting to perform said averaging operations; and setting adegree of signal processing by controlling the rate of said watchedpixel via an ε-filter by setting a gain set value.
 2. An image noisereduction method according to claim 1, further comprising: controllingthe weighting in accordance with the image of an object.
 3. An imagenoise reduction method according to claim 1, further comprising:determining the spatial frequency of an image comprised of the watchedpixels and peripheral pixels; and controlling the weighting inaccordance with the above determination result.
 4. An image noisereduction method according to claim 1, further comprising: digitizingand processing each of the pixel levels.
 5. An image noise reductionapparatus for reducing a noise component, comprising: detection meansfor detecting level differences between a watched pixel and peripheralpixels; selection means for selecting pixels having said leveldifferences smaller than a reference value; operation means forperforming averaging operations by using said selected pixels; weightingmeans for weighting level values of said watched pixel; controllingmeans for controlling the rate of said weighting to perform theaveraging operations by the operation means; and setting means forsetting a degree of signal processing by controlling the rate of saidwatched pixel via an ε-filter by setting a gain set value in said imagenoise reduction apparatus.
 6. An image noise reduction apparatusaccording to claim 5, further comprising: control means for controllingthe weighting in accordance with the image of an object.
 7. An imagenoise reduction apparatus according to claim 5, further comprising:determining means for determining the spatial frequency of an imagecomprised of said watched pixels and peripheral pixels and control meansfor controlling the weighting in accordance with the above determinationresult.
 8. An image noise reduction apparatus according to claim 5,wherein said pixel levels are digitized and processed.